Papers by Christopher D. Manning
Semi-Supervised Sequence Modeling with Cross-View Training (D18-1)
Copied to clipboard
| Challenge: | Unsupervised representation learning algorithms such as word2vec and ELMo only learn from task-specific labeled data during the main training phase. |
| Approach: | They propose a semi-supervised learning algorithm that improves the representations of a Bi-LSTM sentence encoder using a mix of labeled and unlabeled data. |
| Outcome: | The proposed algorithm improves the representations of a Bi-LSTM sentence encoder using a mix of labeled and unlabeled data. |
SLM: Learning a Discourse Language Representation with Sentence Unshuffling (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent models for learning discourse language representations focus on bottom or top-level representations, but they do not capture intermediate-size structures in natural languages such as sentences and the relationships among them. |
| Approach: | They propose a new objective for learning a discourse language representation in a self-supervised manner by shuffling the sequence of input sentences and training a hierarchical transformer model to reconstruct the original ordering. |
| Outcome: | The proposed model improves the original BERT model on downstream tasks by large margins. |
BAM! Born-Again Multi-Task Networks for Natural Language Understanding (P19-1)
Copied to clipboard
| Challenge: | Existing methods to train multi-task neural networks outperform or even match their single-task counterparts are difficult to implement. |
| Approach: | They propose a method that uses knowledge distillation to train multi-task neural networks that outperform or even match their single-task counterparts. |
| Outcome: | The proposed method outperforms or matches single-task neural networks on the GLUE benchmark. |
Finding Universal Grammatical Relations in Multilingual BERT (2020.acl-main)
Copied to clipboard
| Challenge: | Recent work has found that multilingual masked language models learn a surprising amount of linguistic structure, despite a lack of direct linguistic supervision. |
| Approach: | They propose an unsupervised method to find syntactic tree distances in languages other than English and that these subspaces are approximately shared across languages. |
| Outcome: | The proposed method shows that mBERT learns representations of syntactic dependency labels, in the form of clusters, which largely agree with the Universal Dependencies taxonomy. |
A Structural Probe for Finding Syntax in Word Representations (N19-1)
Copied to clipboard
| Challenge: | Existing methods for detecting syntactic knowledge do not test whether syntax trees are embedded in a linear transformation of a neural network’s word representation space. |
| Approach: | They propose a structural probe which evaluates whether syntax trees are embedded in a linear transformation of a neural network’s word representation space. |
| Outcome: | The proposed model shows that entire syntax trees are embedded in deep models’ vector geometry. |
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering (D18-1)
Copied to clipboard
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, Christopher D. Manning
| Challenge: | Existing question answering (QA) datasets fail to train QA systems to perform complex reasoning and provide explanations for answers. |
| Approach: | They propose a new dataset with 113k Wikipedia-based question-answer pairs with four key features: (1) the questions require finding and reasoning over multiple supporting documents to answer; (2) the questions are diverse and not constrained to any pre-existing knowledge bases or knowledge schemas; (3) the questions provide sentence-level supporting facts required for reasoning; and (4) a type of factoid comparison questions to test QA systems’ ability to extract relevant facts and perform necessary comparison. |
| Outcome: | The proposed dataset has 113k Wikipedia-based question-answer pairs and four key features that make it challenging for the latest QA systems. |
Simpler but More Accurate Semantic Dependency Parsing (P18-2)
Copied to clipboard
| Challenge: | Syntactic dependency parsing is the most popular method for automatically extracting low-level relationships between words in a sentence. |
| Approach: | They extend a syntactic dependency parser to train on and generate graph-structured representations that capture between-word relationships that are more closely related to the meaning of a sentence. |
| Outcome: | The proposed system beats the current state-of-the-art system by 0.6% and linguistically richer representations push the margin even higher. |
Universal Dependencies v2: An Evergrowing Multilingual Treebank Collection (2020.lrec-1)
Copied to clipboard
Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Jan Hajič, Christopher D. Manning, Sampo Pyysalo, Sebastian Schuster, Francis Tyers, Daniel Zeman
| Challenge: | Universal Dependencies is an open community effort to create cross-linguistically consistent treebank annotation for many languages. |
| Approach: | They describe version 2 of the universal guidelines and discuss major changes from UD v1 to UD 2 . they propose a morphological layer, a syntactic layer and a word segmentation layer . |
| Outcome: | The proposed treebanks are available for 90 languages and have been updated to meet the needs of multilingual parsers and researchers. |
Stay Hungry, Stay Focused: Generating Informative and Specific Questions in Information-Seeking Conversations (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on question generation assumes knowledge of what the answer might be . instead, questioner must reason pragmatically about how to acquire new information . |
| Approach: | They propose a question generation system that generates pragmatically relevant questions in information-asymmetric conversations. |
| Outcome: | The proposed questioner significantly improves the informativeness and specificity of questions generated over a baseline model as evaluated by metrics as well as humans. |
RNNs can generate bounded hierarchical languages with optimal memory (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that RNNs can efficiently generate bounded hierarchical languages with high syntactic fidelity, but their success is not well-understood theoretically. |
| Approach: | They propose a language of well-nested brackets and m-bounded nesting depth . they prove that an RNN with O(m log k) hidden units suffices, an exponential reduction in memory, by an explicit construction. |
| Outcome: | The proposed language is well-nested brackets and has m-bounded nesting depth . it shows that an RNN with O(m log k) hidden units suffices, an exponential reduction in memory, by an explicit construction. |
Sentences with Gapping: Parsing and Reconstructing Elided Predicates (N18-1)
Copied to clipboard
| Challenge: | Sentences with gapping lack an overt predicate to indicate the relation between two or more arguments. |
| Approach: | They propose two methods for parsing to a Universal Dependencies graph representation that explicitly encodes the elided material with additional nodes and edges. |
| Outcome: | The proposed methods reconstruct elided material from dependency trees with high accuracy when the parser correctly predicts the existence of a gap. |
Pre-Training Transformers as Energy-Based Cloze Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | elucidates close connection between cloze modeling and representation learning over text. |
| Approach: | They propose an energy-based cloze model for representation learning over text . they assign a scalar energy score to each input token indicating how likely it is given context . |
| Outcome: | The proposed model performs better than masked language models and faster than cloze models. |
Graph Convolution over Pruned Dependency Trees Improves Relation Extraction (D18-1)
Copied to clipboard
| Challenge: | Existing dependency-based models neglect crucial information (e.g., negation) by pruning the dependency trees too aggressively. |
| Approach: | They propose an extension of graph convolutional networks that is tailored for relation extraction by pruning dependency trees too aggressively. |
| Outcome: | The proposed model outperforms existing sequence and dependency-based models on the large-scale TACRED dataset and has complementary strengths to sequence models. |
Textual Analogy Parsing: What’s Shared and What’s Compared among Analogous Facts (D18-1)
Copied to clipboard
| Challenge: | Existing methods to extract information from text do not capture disparity between demographic groups. |
| Approach: | They propose a task of Textual Analogy Parsing to model higher-order meanings by comparing poverty rates between different demographic groups. |
| Outcome: | The proposed model can be used to generate graphs from quantitative text. |
Syn-QG: Syntactic and Shallow Semantic Rules for Question Generation (2020.acl-main)
Copied to clipboard
| Challenge: | Question Generation is a simple syntactic transformation but many aspects of semantics influence what questions are good to form. |
| Approach: | They propose a set of syntactic rules which transform declarative sentences into question-answer pairs. |
| Outcome: | The proposed system generates a larger number of highly grammatical and relevant questions than existing QG systems. |
Stanza: A Python Natural Language Processing Toolkit for Many Human Languages (2020.acl-demos)
Copied to clipboard
| Challenge: | Existing tools that support only a few major languages are under-optimized for accuracy due to a focus on efficiency or use of less powerful models. |
| Approach: | They introduce a Python natural language processing toolkit that supports 66 languages . they train Stanza on 112 datasets and show it generalizes well on all languages compared to other tools . |
| Outcome: | The proposed toolkit performs well on 112 datasets and is compatible with the popular Java CoreNLP software. |
Answering Complex Open-domain Questions Through Iterative Query Generation (D19-1)
Copied to clipboard
| Challenge: | Currently, one-step retrieve-and-read question answering systems cannot answer such questions because they rarely contain retrievable clues about the missing entity. |
| Approach: | They propose a multi-step approach to retrieve relevant content with the question, then reading the paragraphs returned by the information retrieval component to arrive at the final answer. |
| Outcome: | The proposed model outperforms the best previously published model despite not using pretrained language models such as BERT. |
Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reports (2020.acl-main)
Copied to clipboard
| Challenge: | Existing abstractive summarization models do not guarantee factual correctness of summaries . |
| Approach: | They propose a framework where models evaluate factual correctness by fact-checking it against its reference using an information extraction module. |
| Outcome: | The proposed method significantly improves the factual correctness and overall quality of outputs over a competitive neural summarization system, producing radiology summaries that approach the quality of human-authored ones. |